The following scripts are related to both the Integration Point: Response and the Integration Point: Analysis. Click on the links for more information about these integration points.
This example demonstrates how to compute the probability of success of a clinical trial by extending East Horizon’s single-endpoint framework to handle dual endpoints using custom R scripts for the Response (Patient Simulation) and the Analysis integration points. This example uses Progression Free Survival and Overall Survival as endpoints, however, this could be extended to other types of endpoints by modifying the R code.
To compute the Probability of Success of a trial, users need to simulate patient outcomes by sampling the true rates from prior distributions. This is something that East Horizon cannot handle yet with its current response generation algorithms, so we must integrate a custom R file for the simulation to do so.
When a trial involves multiple success criteria, users must change how “Success” is defined from a traditional success analysis with only one endpoint. East Horizon cannot handle multiple success criteria for a Winning Condition yet, so we must integrate a custom R file for the simulation to take into account changing success criteria from the traditional “statistically significant” criteria.
In the R directory of this example you will find the following R files:
Simulate2EndpointTTEWithMultiState.R
This R file contains a function that executes the response data generation for both PFS and OS endpoints. Due to the typically assumed correlation between PFS and OS, the timing of the events for each simulated patient must take into account that these endpoints are not completely independent of each other. This function therefore handles this concern by using a multi-state model that takes into consideration:
As there may be uncertainty around each of these assumed parameters, the function simulates each of them based on prior distributions rather than single values for each. Users must provide input assumptions that parametrize the prior distributions that will be sampled from to simulate each patient’s response times.
For more information on this function, see the Response (Patient Simulation) Integration Point section below.
This R file contains a function that executes the interim and final analyses of each simulated trial. The function takes into account that both PFS and OS event time stamps are being generated. The function computes the probability of success of the trial, where success is defined as:
The success criteria can be changed to compute the probability of success for varying success criteria. The lines in the code that need to be modified are line 170 for interim analyses and line 193 for final analysis.
For more information on this function, see the Analysis Integration Point section below.
The figure below illustrates where this example fits within the R integration points of Cytel products, accompanied by flowcharts outlining the general steps performed by the R code.
By combining the above R functions with the East Horizon native inputs, users are able to simulate and compute the probability of success of each simulated trial. Users will also continue to benefit from East Horizon’s output visualizations – with the caveat that the Probability of Success metric will be labeled as “Power” in the native outputs of East Horizon.
Before starting, make sure you have the required tools and files.
These variables are the thresholds for the Hazard Ratio for Overall Survival to be considered “TRENDING” for the trial’s second success criteria. Users must specify a threshold for the interim analysis, if using a group sequential design, and always for the final analysis. See the Analysis Integration Point section below for more information about these variables and example values.
Note: We typically make success criteria more CONSERVATIVE (i.e. making it more difficult to declare a “win”) on a trending OS at the interim analysis compared to the final analysis because you want to be “more sure” of your decision when you have less evidence/data available at the time of decision.
See the Response (Patient Simulation) Integration Point section below for more information about these variables and example values.
Enter your desired mean and variance to calculate the shape and rate parameters for the Gamma distribution:
Shape: -
Rate: -
See the Response (Patient Simulation) Integration Point section below for more information on these variables and example values.
See the Results section below for more information.
This endpoint is related to this R file: Simulate2EndpointTTEWithMultiState.R
We are using East Horizon’s single-endpoint framework, which we customize to support dual endpoints through the Response Integration Point via our R script linked above. The two endpoints of interest are:
We use a multi-state model to simulate event times for each patient in every simulated trial. This model captures the relationship between PFS and OS by generating both outcomes together, rather than independently. Information generated from this simulation will be used later for the Analysis Integration Point.
The diagram above illustrates the multistate model for survival analysis, with three states:
There are also three transitions between states:
From Meller et al. [1], the probability of progression before death is:
\[ q = \frac{\alpha_{01}}{\alpha_{01} + \alpha_{02}} \]
Where \(q = 1 - p\), with \(p\) being the probability of Death before Progression.
Using this relationship:
\[ \alpha_{02} = \frac{\alpha_{01} \cdot p}{q} \]
Or equivalently:
\[ \alpha_{01} = \frac{\alpha_{02} \cdot q}{p} \]
Progression-Free Survival (PFS) is treated as the minimum of two competing times:
Using the property of the minimum of two exponential random variables [2]:
\[ Z = \min(X_1, X_2) \sim \text{Exp}(\alpha_{01} + \alpha_{02}) \]
The median of an exponential distribution is given by:
\[ M = \frac{\log(2)}{\alpha_{01} + \alpha_{02}} \]
Substituting \(\alpha_{01}\) in terms of \(\alpha_{02}\):
\[ M = \frac{1}{\alpha_{01} + \alpha_{02}} = \frac{1}{\frac{\alpha_{02} \cdot q}{p} + \alpha_{02}} = \frac{1}{\alpha_{02} \cdot \left(\frac{q}{p} + 1\right)} \]
Solving for \(\alpha_{02}\):
\[ \alpha_{02} = \frac{\log(2)}{M \cdot \left(\frac{q}{p} + 1\right)} \]
Once \(\alpha_{02}\) is calculated, \(\alpha_{01}\) can be derived using:
\[ \alpha_{01} = \frac{\alpha_{02} \cdot q}{p} \]
Overall Survival (OS) accounts for both pathways:
The median OS (\(M_{OS}\)) is approximated as a weighted combination of these two pathways:
\[ M_{OS} = \text{Median}(X_2 \cdot p + (X_1 + X_3) \cdot q) = p \cdot \frac{\log(2)}{\alpha_{02}} + q \cdot (\frac{\log(2)}{\alpha_{01}} + \frac{\log(2)}{\alpha_{12}})\]
To compute \(\alpha_{12}\), the
model numerically solves for the rate using uniroot,
ensuring consistency with the user-provided median OS.
Sources
One option is to directly input the median times and probabilities of death before progression into the script as user parameters. Refer to the table below for the definitions and example values of these user-defined parameters.
| User parameter | Definition | Value |
|---|---|---|
| dMedianPFS0 | Median time to PFS event for control group | 12 |
| dMedianPFS1 | Median time to PFS event for treatment group | 18 |
| dMedianOS0 | Median time to OS event for control group | 18 |
| dMedianOS1 | Median time to OS event for treatment group | 27 |
| dProbOfDeathBeforeProgression0 | Probability of death before PFS for control group | 0.2 |
| dProbOfDeathBeforeProgression1 | Probability of death before PFS for treatment group | 0.2 |
Here, the median times are in months and the hazard ratio for both endpoints (PFS & OS) is equal to \(\frac{12}{18} = \frac{18}{27} = 0.6667\). The probability of death before progression is 20% for both control and treatment arms.
Another option is to customize how patient data is simulated by building a more realistic model for both PFS and OS outcomes using prior distributions instead of directly using median times and probabilities. Using prior distributions allows us to account for uncertainty around the true treatment effect, which enables users to identify the Probability of Success of a trial rather than statistical Power. Here’s how the event data is generated:
All three parameters are treated as random variables, sampled from prior distributions, allowing each simulated trial to reflect a range of possible real-world outcomes.
Refer to the table below for the definitions of the user-defined parameters used in this example.
| User Parameter | Definition |
|---|---|
| dMedianPFS0PriorShape | Shape parameter for the median time to PFS event for control group |
| dMedianPFS0PriorRate | Rate parameter for the median time to PFS event for control group |
| dMedianPFS1PriorShape | Shape parameter for the median time to PFS event for treatment group |
| dMedianPFS1PriorRate | Rate parameter for the median time to PFS event for treatment group |
| dMedianOS0PriorShape | Shape parameter for the median time to OS event for control group |
| dMedianOS0PriorRate | Rate parameter for the median time to OS event for control group |
| dMedianOS1PriorShape | Shape parameter for the median time to OS event for treatment group |
| dMedianOS1PriorRate | Rate parameter for the median time to OS event for treatment group |
| dProbOfDeathBeforeProgression0Param1 | Alpha parameter for probability of death before progression for control group |
| dProbOfDeathBeforeProgression0Param2 | Beta parameter for probability of death before progression for control group |
| dProbOfDeathBeforeProgression1Param1 | Alpha parameter for probability of death before progression for treatment group |
| dProbOfDeathBeforeProgression1Param2 | Beta parameter for probability of death before progression for treatment group |
The shape and rate parameters can be calculated from the assumed mean (i.e. median survival time) and its variance. You can use the Parameter Solving tool from the Step-by-Step section above to compute these required parameters.
If you have the scale parameter for your assumption instead of the rate parameter, you can convert from one to the other using the formula \(Rate = \frac{1}{Scale}\).
In this first example scenario, we want:
Using the tool above, we get:
Refer to the table below for the values of all user-defined parameters used in this example.
| User parameter | Value |
|---|---|
| dMedianPFS0PriorShape | 14.4 |
| dMedianPFS0PriorRate | 1.2 |
| dMedianPFS1PriorShape | 32.4 |
| dMedianPFS1PriorRate | 1.8 |
| dMedianOS0PriorShape | 32.4 |
| dMedianOS0PriorRate | 1.8 |
| dMedianOS1PriorShape | 72.9 |
| dMedianOS1PriorRate | 2.7 |
| dProbOfDeathBeforeProgression0Param1 | 20 |
| dProbOfDeathBeforeProgression0Param2 | 80 |
| dProbOfDeathBeforeProgression1Param1 | 20 |
| dProbOfDeathBeforeProgression1Param2 | 80 |
In this second example scenario, we want:
Using the tool above, we get:
Refer to the table below for the values of all user-defined parameters used in this example.
| User parameter | Value |
|---|---|
| dMedianPFS0PriorShape | 14.4 |
| dMedianPFS0PriorRate | 1.2 |
| dMedianPFS1PriorShape | 16.2 |
| dMedianPFS1PriorRate | 0.9 |
| dMedianOS0PriorShape | 32.4 |
| dMedianOS0PriorRate | 1.8 |
| dMedianOS1PriorShape | 36.45 |
| dMedianOS1PriorRate | 1.35 |
| dProbOfDeathBeforeProgression0Param1 | 20 |
| dProbOfDeathBeforeProgression0Param2 | 80 |
| dProbOfDeathBeforeProgression1Param1 | 20 |
| dProbOfDeathBeforeProgression1Param2 | 80 |
In this third example scenario, we want:
Using the tool above, we get:
Refer to the table below for the values of all user-defined parameters used in this example.
| User parameter | Value |
|---|---|
| dMedianPFS0PriorShape | 14.4 |
| dMedianPFS0PriorRate | 1.2 |
| dMedianPFS1PriorShape | 14.4 |
| dMedianPFS1PriorRate | 1.2 |
| dMedianOS0PriorShape | 32.4 |
| dMedianOS0PriorRate | 1.8 |
| dMedianOS1PriorShape | 32.4 |
| dMedianOS1PriorRate | 1.8 |
| dProbOfDeathBeforeProgression0Param1 | 20 |
| dProbOfDeathBeforeProgression0Param2 | 80 |
| dProbOfDeathBeforeProgression1Param1 | 20 |
| dProbOfDeathBeforeProgression1Param2 | 80 |
In this final example scenario, we want:
Using the tool above, we get:
Refer to the table below for the values of all user-defined parameters used in this example.
| User parameter | Value |
|---|---|
| dMedianPFS0PriorShape | 14.4 |
| dMedianPFS0PriorRate | 1.2 |
| dMedianPFS1PriorShape | 7.2 |
| dMedianPFS1PriorRate | 0.6 |
| dMedianOS0PriorShape | 32.4 |
| dMedianOS0PriorRate | 1.8 |
| dMedianOS1PriorShape | 16.2 |
| dMedianOS1PriorRate | 0.9 |
| dProbOfDeathBeforeProgression0Param1 | 20 |
| dProbOfDeathBeforeProgression0Param2 | 80 |
| dProbOfDeathBeforeProgression1Param1 | 20 |
| dProbOfDeathBeforeProgression1Param2 | 80 |
This endpoint is related to this R file: AnalyzePFSAndOS.R
Using the file above, the Analysis element of East Horizon’s simulation is customized to compute the probability of success (PoS) for the trial, based on dual endpoints: Progression-Free Survival (PFS) and Overall Survival (OS). The file uses information from the simulation (SimData variable) that is generated by the Response element of East Horizon’s simulation. See the Response section above for more information about the PFS and OS endpoints generation.
The criteria for declaring trial success are as follows. Both criteria must be met to declare success:
Note: At an interim analysis, we usually set stricter criteria for a positive trend on OS compared to the final analysis. This is because we have less data early on, so we want to be more confident in any decision we make at that stage. Therefore, the threshold defining a positive trend at the interim is typically lower than the threshold used at the final analysis.
Refer to the table below for the definitions of the user-defined parameters used in this example.
| User parameter | Definition |
|---|---|
| HazardRatioCutoffIA | OS hazard ratio threshold used for the interim analysis |
| HazardRatioCutoffFA | OS hazard ratio threshold used for the final analysis |
The first example option is a fixed sample design with a hazard ratio threshold of 0.9. This option shows that we can still use this script without interim analyses. Refer to the table below for the values of the user-defined parameters used in this option.
| User parameter | Value |
|---|---|
| HazardRatioCutoffFA | 0.9 |
The second example option is a group sequential design with hazard ratio thresholds of 0.8 (for interim analysis) and 0.9 (for final analysis). Refer to the table below for the values of the user-defined parameters used in this option.
| User parameter | Value |
|---|---|
| HazardRatioCutoffIA | 0.8 |
| HazardRatioCutoffFA | 0.9 |
The third example option is a group sequential design with a lower hazard ratio threshold of 0.5 for both interim and final analyses. Refer to the table below for the values of the user-defined parameters used in this option.
| User parameter | Value |
|---|---|
| HazardRatioCutoffIA | 0.5 |
| HazardRatioCutoffIA | 0.5 |
The fourth example option is a group sequential design with a lower hazard ratio threshold of 0.5 (for interim analysis) and a higher hazard ratio threshold of 1.2 (for final analysis). Refer to the table below for the values of the user-defined parameters used in this option.
| User parameter | Value |
|---|---|
| HazardRatioCutoffIA | 0.5 |
| HazardRatioCutoffIA | 1.2 |
In the Results section, Power now refers to the probability of success, where success is defined as a statistically significant difference in time to progression-free survival (PFS) between the control and treatment arm as well as a positive trend in overall survival (OS), i.e. the time to overall survival (OS) is longer in magnitude for the patients in the treatment arm compared to those in the control arm. Below is an example of the heatmap that could be generated by East Horizon following the simulation. Each square represents the simulated probability of success for a trial, based on a specific combination of Response scenario (columns) and Analysis option (rows).